Data and Workload Distribution in a Multithreaded Architecture

Matching data distribution to workload distribution is important to improve the performance of distributedmemory multiprocessors. While data and workload distribution can be tailored to fit a particular problem to a particular distributed-memory architecture, it is often difficult to do so for various reasons including complexity of address computation, runtime data movement, and irregular resource usage. This report presents our study on multithreading for distributed-memory multiprocessors. Specifically, we investigate the effects of multithreading on data distribution and workload distribution with variable thread granularity. Various types of workload distribution strategies are defined along with thread granularity. Several types of data distribution strategies are investigated. These include row-wise cyclic, k-way partial-row cyclic, and blocked distribution. To investigate the performance of multithreading, two problems are selected: highly sequential Gaussian Elimination with Partial Pivoting and highly parallel Matrix Multiplication. Execution results on the 80-processor EM-4 distributed-memory multiprocessor indicate that multithreading can offset the loss that is due to the mismatch between data distribution and workload distribution even for sequential and irregular problems while giving high absolute performance. 1 Dept. of Computer and Information Science, New Jersey Institute of Technology, Newark, NJ 07102-1982, [email protected] 2 Computer Architecture Section, Electrotechnical Laboratory, 1-1-4 Umezono, Tsukuba-shi, Ibaraki 305, Japan, [email protected] 3 Dept. of EE-Systems, EEB336, University of Southern California, Los Angeles, California 90089-2563, {nyoo, gaudiot}@usc.edu